Automatic linearity detection

Given a function, or more generally an operator, the question "Is it linear?" seems simple to answer. In many applications of scientific computing it might be worth determining the answer to this question in an automated way; some functionality, such as operator exponentiation, is only defined for linear operators, and in other problems, time saving is available if it is known that the problem being solved is linear. Linearity detection is closely connected to sparsity detection of Hessians, so for large-scale applications, memory savings can be made if linearity information is known. However, implementing such an automated detection is not as straightforward as one might expect. This paper describes how automatic linearity detection can be implemented in combination with automatic differentiation, both for standard scientific computing software, and within the Chebfun software system. The key ingredients for the method are the observation that linear operators have constant derivatives, and the propagation of two logical vectors, $\ell$ and $c$, as computations are carried out. The values of $\ell$ and $c$ are determined by whether output variables have constant derivatives and constant values with respect to each input variable. The propagation of their values through an evaluation trace of an operator yields the desired information about the linearity of that operator

Automatic Frechet differentiation for the numerical solution of boundary-value problems

A new solver for nonlinear boundary-value problems (BVPs) in Matlab is presented, based on the Chebfun software system for representing functions and operators automatically as numerical objects. The solver implements Newton's method in function space, where instead of the usual Jacobian matrices, the derivatives involved are Frechet derivatives. A major novelty of this approach is the application of automatic differentiation (AD) techniques to compute the operator-valued Frechet derivatives in the continuous context. Other novelties include the use of anonymous functions and numbering of each variable to enable a recursive, delayed evaluation of derivatives with forward mode AD. The AD techniques are applied within a new Chebfun class called chebop which allows users to set up and solve nonlinear BVPs in a few lines of code, using the "nonlinear backslash" operator (\). This framework enables one to study the behaviour of Newton's method in function space

Numerical solution of nonlinear boundary value problems for ordinary differential equations in the continuous framework

Ordinary differential equations (ODEs) play an important role in mathematics. Although intrinsically, the setting for describing ODEs is the continuous framework, where differential operators are considered as maps from one function space to another, common numerical algorithms for ODEs discretise problems early on in the solution process. This thesis is about continuous analogues of such discrete algorithms for the numerical solution of ODEs. This thesis shows how Newton's method for finite dimensional system can be generalised to function spaces, where it is known as Newton-Kantorovich iteration. It presents affine invariant damping strategies for increasing the chance of convergence for the Newton-Kantorovich iteration. The derivatives required in this continuous setting are Fréchet derivatives, the continuous analogue of Jacobian matrices. In this work, we present how automatic differentiation techniques can be applied to compute Fréchet derivatives. We introduce chebop, a Matlab solver for nonlinear boundary-value problems, which combines damped Newton iteration in function space and automatic Fréchet differentiation. By proving that affine operators have constant Fréchet derivatives, it is demonstrated how automatic linearity detection of computed quantities can be implemented. This is valuable for black-box solvers, which can use the information to determine whether an iteration scheme has to be employed for solving a problem. Like nonlinear systems of equations, nonlinear boundary-value problems can have multiple solutions. This thesis present two techniques for obtaining multiple solutions of operator equations: deflation and path-following. An algorithm combining the two techniques is proposed.</p

Numerical solution of nonlinear boundary value problems for ordinary differential equations in the continuous framework

Ordinary differential equations (ODEs) play an important role in mathematics. Although intrinsically, the setting for describing ODEs is the continuous framework, where differential operators are considered as maps from one function space to another, common numerical algorithms for ODEs discretise problems early on in the solution process. This thesis is about continuous analogues of such discrete algorithms for the numerical solution of ODEs. This thesis shows how Newton's method for finite dimensional system can be generalised to function spaces, where it is known as Newton-Kantorovich iteration. It presents affine invariant damping strategies for increasing the chance of convergence for the Newton-Kantorovich iteration. The derivatives required in this continuous setting are Fr&eacute;chet derivatives, the continuous analogue of Jacobian matrices. In this work, we present how automatic differentiation techniques can be applied to compute Fr&eacute;chet derivatives. We introduce chebop, a Matlab solver for nonlinear boundary-value problems, which combines damped Newton iteration in function space and automatic Fr&eacute;chet differentiation. By proving that affine operators have constant Fr&eacute;chet derivatives, it is demonstrated how automatic linearity detection of computed quantities can be implemented. This is valuable for black-box solvers, which can use the information to determine whether an iteration scheme has to be employed for solving a problem. Like nonlinear systems of equations, nonlinear boundary-value problems can have multiple solutions. This thesis present two techniques for obtaining multiple solutions of operator equations: deflation and path-following. An algorithm combining the two techniques is proposed.This thesis is not currently available in OR